This invention relates to the digital manipulation of video images, for example in the creation of works to be broadcast or distributed as films or video records on other media such as discs or tapes.
There are many ways of digitally manipulating images. One type of manipulation includes at the xe2x80x98mattingxe2x80x99 of image components. Typically these techniques involve the use of a camera to record an actor, or other subject, in front of a background of an even colour. This colour is often blue, hence the generic description of these techniques as xe2x80x98Bluescreenxe2x80x99 techniques. The picture signal is then analysed to be either pixels of blue [where the background is] or not blue, where the subject is. This digital selector map is referred to as a xe2x80x98keyxe2x80x99 image. Another image is then selected, such as an outdoor scene. By using the key image to select between the two images, it is possible to create a composite image of the foreground actor against the chosen background. These techniques can be performed on motion picture sequences, by processing serially each frame. The Ulitimatte Corporation produce systems that perform this function, and has done so for many years. These systems are well known for use in producing images which appear xe2x80x9crealisticxe2x80x9d.
A second mode of creating images is using a xe2x80x983-Dxe2x80x99 modelling system. Such systems are available from many manufacturers, including the Canadian company Softimage, and the xe2x80x983-D Studio-Maxxe2x80x99 product from Kinetix, who are a division of Autodisk, a US Corporation. With these products, scenes are constructed as xe2x80x983-Dxe2x80x99 models, thus specifying a geometrical arrangement between all of the 3-D items in the scene both in orientation and in distance. A xe2x80x98cameraxe2x80x99 view is then specified, together with a camera xe2x80x98trackxe2x80x99 and then the resultant view calculated on a frame by frame basis. These systems are well known for producing an xe2x80x98unrealxe2x80x99 look, and are often used to give xe2x80x98fantasyxe2x80x99 or xe2x80x98science fictionxe2x80x99 style images.
GB-A2317299 of Pandora International Limited discloses techniques for utilising a digital colour processing system, such as Pandora""s POGLE DCP system, to locate areas of picture for changing. For example, consider the processing of a real scene which contains a brick wall. GB-A2317299 discloses techniques to locate the area of brick wall, using known parameters of brick wall texture, and brick wall, reddish-brown. Once this area has been detected, there are disclosed techniques to produce items of other textures in its place, such as a marble wall. The marble texture can be generated using the xe2x80x98OpenGLxe2x80x99 graphics descriptor language, as produced by The Silicon Graphics Incorporated, of Mountain View, Calif., USA.
Whilst these techniques are very useful, there are limitations and problems may occur with these methods under certain circumstances. Consider a scene that contains a brick wall, a street lamp, a post box, and a red car. The brick wall looks a natural part of the scene because of the ambient lighting affects. These include the light falling on the brick wall, making a higher luminance at one end of the wall than the other. There may also be a shadow of the post box which falls on the brick wall, causing a definite geometric region of the wall where there is less luminance. If the car is brightly lit, and is of a xe2x80x98shinyxe2x80x99 appearance, the car may also reflect some red light onto the wall.
Simply using the techniques disclosed in GB-A-2317299 would result in the replacement of this brick wall with a flat lit wall of, say, marble texture. Because of the loss of the uneven light on the wall, the xe2x80x98shadowxe2x80x99 of the post box, and the reflections of the red car, the composite image would look artificial.
Viewed from one aspect, the prevention invention provides a method for the digital manipulation of a video image, comprising the steps of locating a first object in the image, determining a base set of appearance parameter values for the first object, analysing the first object by selected regions and in respect of each selected region of the first object determining the extent to which actual appearance parameter values deviate from the base act of appearance parameter values, storing data representative of the positions of the regions and the respective deviations from the base set of values, replacing the first object by a second object, and applying to regions of the second object adjustments to appearance parameter values by reference to the stored data concerning the positions of the regions and the respective deviations from the base set of appearance parameter values.
Thus, the second object can be adjusted to have areas of increased light, shadows, reflected light and so forth that were apparent on the original object so as to provide a more lifelike affect. The second object may be an entirely different object, for example the replacement of one person for another or one type of car for another. It could be a real image previously recorded or a computer generated image. Alternatively the second object could be the first image to which a different texture or the like has been appliedxe2x80x94such as the conversion of a brick wall to a marble wall as discussed above. The regions of the first object which are analysed and the regions of the second object which are adjusted will normally be the pixels making up the object, although either operation could be done on groups of pixels. The adjustment of the appearance parameters values for the second object could be done after the second object has been placed into the image, or beforehand or simultaneously.